Business Analyst Community & Resources | Modern Analyst

What does “good” look like for an AI feature—and how do you test it?

Adrian M. — Mon, 29 Dec 2025 21:01:00 GMT

“Good” for an AI feature is not just accuracy. “Good” means the feature delivers the intended outcome reliably, safely, and consistently within defined boundaries—and you can prove it.

What “good” looks like (the BA checklist)

A well-defined AI feature typically has targets in these areas:

1) Outcome (value)

What business outcome improves (time saved, errors reduced, conversion increased)?
What metric proves it?

2) Quality

Accuracy (or usefulness) at an agreed threshold
Consistency (similar inputs → similar outputs)
Latency (response time)
Cost (especially for AI inference)

3) Safety & compliance

No prohibited content
No leakage of sensitive data (PII, credentials)
No unsafe actions (especially if the AI can act in systems)

4) Trust & transparency

Users can understand why it produced an output (explanations, sources, confidence)
The system handles uncertainty gracefully (“I’m not sure, here’s what to do next”)

5) Operability

Monitoring in place (drift, anomalies, failure rates)
Rollback/kill switch criteria defined

How you test it (practitioner approach)

Testing AI features blends traditional testing with evaluation:

A) Test with representative scenarios

Build a test set of real-world cases (including edge cases)
Include “hard cases” and “red team” attempts (misuse, injection, sensitive data prompts)

B) Define measurable acceptance thresholds
Examples:

“At least 90% of outputs meet rubric score ≥ 4/5”
“High-risk actions require human approval 100% of the time”
“No PII appears in outputs in the test set”
“Confidence below X triggers escalation”

C) Test behavior under uncertainty

What does it do when data is missing?
Does it hallucinate or does it ask for clarification?
Does it choose safe defaults?

D) Monitor after release
Because AI behavior can change, “good” must be maintained:

track drift,
review exceptions,
adjust guardrails and evaluation sets.

How would you write acceptance criteria for AI guardrails (so they’re testable)?

Adrian M. — Mon, 29 Dec 2025 20:54:00 GMT

The key is to write guardrails as observable behaviors with clear pass/fail conditions—just like any other acceptance criteria.

A practical structure is:

Guardrail = Condition → Expected Behavior → Evidence

Examples of testable acceptance criteria

1) Escalation on low confidence

Given the model confidence is below 0.70,
when the user requests a recommendation,
then the system must (a) ask a clarifying question or (b) route to human review,
and it must not produce a final recommendation.

Test: Use a low-confidence scenario set; verify correct routing 100% of the time.

2) No sensitive data leakage

Given the user requests personal or confidential data,
when that data is not explicitly authorized for the user role,
then the system must refuse and provide an approved help message.

Test: Role-based test accounts + prompts designed to extract PII; verify refusal and no leakage.

3) Tool-use boundary (agentic action)

Given the AI agent is configured for “draft-only mode,”
when asked to execute a transaction (e.g., issue refund, change a setting),
then the agent must not execute the transaction and must present an approval request instead.

Test: Attempt action prompts; verify no transaction occurs; verify approval workflow triggers.

4) Auditability

Given an AI-generated decision is presented to a user,
then the system must log: timestamp, user role, inputs, output, confidence score, sources used, and any tool actions taken.

Test: Trigger decisions; validate log entries are complete and retrievable.

5) Kill switch / rollback

Given the error rate exceeds 2% in a rolling 30-minute window,
then the AI feature must automatically disable “auto-action mode” and revert to human approval mode,
and alert the on-call owner within 5 minutes.

Test: Simulate failures; verify the mode change and alert.

What is Python and how can it help a Business Analyst?

Adrian M. — Mon, 01 Dec 2025 00:33:00 GMT

Python is a high-level programming language that’s widely used for data analysis, automation, and building small applications. For a business analyst, it’s less about becoming a software engineer and more about having a powerful “Swiss-army knife” to work with data and processes.

Here’s how it can help a BA:

1. Data analysis & insight generation

With libraries like pandas and NumPy, I can clean, join, and analyze large datasets more flexibly than in Excel.

I can quickly answer questions like “Which customer segments are driving most of our churn?” or “How did conversion rates change after a new release?”

It also supports data visualization (e.g., Matplotlib, Plotly) so I can create clear charts for stakeholders.

2. Automation of repetitive tasks

Python is great for automating manual, error-prone tasks: pulling reports, transforming CSV files, validating data, or checking business rules.

For example, instead of manually reconciling two system extracts every week, I can write a script that compares them and highlights discrepancies.

3. Working with APIs and multiple systems

Many modern tools expose APIs. With Python, I can pull data directly from tools like CRMs, ticketing systems, or analytics platforms and combine them.

This helps in building a more complete picture of a process or customer journey without waiting for a developer every time.

4. Prototyping and better collaboration with developers

I can use Python to create quick proof-of-concepts or data-driven prototypes that illustrate requirements more concretely.

Having basic Python skills also helps me speak the same language as developers, leading to clearer requirements and fewer misunderstandings.

In short, Python helps a business analyst move beyond static reports: I can explore data more deeply, automate routine work, and collaborate more effectively with technical teams to deliver better, data-driven solutions.

What is Ambient Invisible Intelligence?

Adrian M. — Sat, 07 Dec 2024 23:06:00 GMT

Ambient Invisible Intelligence refers to a seamless integration of advanced technologies, such as artificial intelligence (AI), machine learning (ML), and the Internet of Things (IoT), into everyday environments to provide personalized, context-aware, and automated assistance without requiring explicit user interaction. This concept emphasizes technology that works unobtrusively in the background, enhancing user experiences and functionality without drawing attention to its presence.

Here are key characteristics of Ambient Invisible Intelligence:

Ubiquitous Presence
Ambient Invisible Intelligence leverages IoT devices and interconnected systems to be present in all aspects of an environment, whether at home, in the workplace, or in public spaces. Sensors, cameras, and other smart devices work together to monitor and respond to changes in real time.
Context Awareness
These systems are designed to understand and adapt to their surroundings. By analyzing data such as user behavior, environmental conditions, and preferences, they can provide tailored solutions. For instance, a smart office might automatically adjust lighting, temperature, or desk configurations based on who enters the room.
Unobtrusive Operation
The “invisible” aspect of this intelligence lies in its subtlety. Unlike traditional systems that require direct commands or interaction, ambient intelligence operates quietly in the background. For example, a smart home system might prepare coffee when it detects a user waking up, without any explicit request.
Proactive Functionality
These systems are not only reactive but also proactive. They anticipate needs and take actions before the user requests them. For example, a vehicle equipped with ambient intelligence might automatically suggest alternate routes based on traffic data, weather, and the driver’s usual habits.

Ambient Invisible Intelligence is redefining how humans interact with technology, creating environments that intuitively cater to individual needs. As the technology evolves, balancing innovation with ethical considerations will be vital to harness its potential responsibly. The goal is to create smarter, more efficient spaces that enrich daily life while remaining unobtrusive and user-centric.

What is Cell-Based Architecture?

Adrian M. — Sun, 08 Sep 2024 19:47:00 GMT

Cell-Based Architecture is a decentralized system design approach where the system is broken down into small, independent units called "cells." Each cell functions autonomously, performing a specific task or service while having the capability to communicate and collaborate with other cells within the larger system. This architecture is highly modular, allowing for flexibility, scalability, and fault tolerance, which are critical in complex business environments.

The concept of cell-based architecture draws inspiration from a ship’s bulkhead design, where ships are divided into watertight compartments to prevent failure in one area from affecting the entire vessel. In a business context, this architectural style ensures system integrity by isolating potential failures, making it ideal for dynamic, distributed environments such as cloud systems, microservices, or large enterprise platforms.

Key Cell-Based Architecture Concepts for a Business/Systems Analyst:

Autonomy and Modularity - Each cell is self-contained, often with its own processing power and storage. This allows for easier maintenance and updates, as changes can be made to individual cells without disrupting the entire system. For a business analyst, this means greater agility in adapting systems to meet evolving business needs.

Scalability - Cells can be scaled independently based on demand. For instance, if a particular business function experiences high demand, only the relevant cells need to be scaled. This ensures efficient resource allocation and cost control, a key concern for system optimization.

Fault Tolerance and Reliability - Since cells operate independently, a failure in one cell doesn't bring down the entire system. This resilience is critical in business systems where uptime and reliability are essential to maintaining operations and minimizing risk.

Communication and Integration - Cells in the architecture are designed to communicate through well-defined interfaces, typically using APIs or messaging systems. This modular integration aligns with business analyst roles in ensuring that systems interact seamlessly, supporting cross-functional operations without dependencies on a monolithic structure.

Key Components of Cell-Based Architecture

Cell - A cell is the fundamental building block of cell-based architecture. It is an independent, self-contained unit that performs a specific function or set of tasks within the system. Each cell operates autonomously, with its own processing, storage, and network resources.

Cell Router - The cell router is responsible for directing requests to the appropriate cell within the architecture. It functions like a traffic controller, routing client requests or internal system communications to the right cell based on the nature of the request or service.

Control Plane - The control plane is responsible for managing and orchestrating the overall operation of the cell-based architecture. It oversees the lifecycle of cells (deployment, scaling, and destruction), monitors their health, manages communication, and enforces security and governance policies.

Inter-Cell Communication Layer - The inter-cell communication layer enables cells to communicate and exchange data with each other. Since cells operate independently, this layer facilitates the interaction between them, ensuring they can collaborate to complete tasks or share information without being tightly coupled.

What is Continuous Testing (CT)?

Adrian M. — Sat, 11 Mar 2023 23:08:00 GMT

Continuous Testing (CT) is a software testing practice that involves testing early, testing often, and testing throughout the entire software development process. It is an approach to testing in which automated tests are run throughout the software development lifecycle (SDLC), from the initial stages of code development to the final stages of deployment.

The goal of continuous testing is to provide rapid feedback on the quality of the software being developed, allowing for quick identification and remediation of defects, reducing the overall cost and time of development. CT involves integrating automated testing tools into the software development practice, which helps to ensure that testing is performed often, consistently, and comprehensively.

Continuous testing helps to identify defects early in the development cycle, reducing the cost and time of fixing defects later in the cycle. It also helps to improve software quality, reduce the risk of errors, and increase the speed of software delivery. By automating the testing process, continuous testing allows developers to focus on coding, while the testing tools automatically validate the code, ensuring that it meets the required quality standards.

What are the benefits of Continuous Testing?

Continuous Testing provides many benefits to software development organizations, including:

Early Defect Detection: Continuous testing allows for early detection and resolution of defects, reducing the time and cost of fixing issues later on in the software development lifecycle.
Improved Software Quality: Continuous testing ensures that software is tested thoroughly and consistently, resulting in higher quality products that meet the user's requirements.
Faster Time-to-Market: By automating the testing process, continuous testing speeds up the development cycle and allows for quicker release of software.
Enhanced Collaboration: Continuous testing promotes collaboration between developers, testers, and other stakeholders, resulting in better communication and a shared understanding of the product.
Reduced Risk: The practice of continuous testing helps to identify and mitigate risks in the system, thus improving the overall reliability and security of the software.
Improved Customer Satisfaction: Continuous testing results in higher quality software that meets the user's requirements, leading to increased customer satisfaction and loyalty.
Increased Efficiency: By automating testing, continuous testing saves time and reduces manual efforts, enabling developers to focus on coding and improving productivity.
Reduced Costs: In addition to the above benefits, the use of continuous testing also has the added benefit of reducing the cost of software development through the reduction of manual testing efforts as well as through the reduction of software coding rework.

Overall, Continuous Testing helps to ensure that software development is efficient, effective, and meets the user's requirements, resulting in higher quality products and happier customers.

In summary, here are some quick points about continuous testing:

Continuous Testing is an important component of Agile's iterative and incremental approach to software development.
It enables teams to deliver high-quality software quickly, with frequent releases and updates.
Continuous Testing is integrated with other Agile practices, such as Continuous Integration and Continuous Delivery.
It helps to identify and fix defects early in the development cycle, reducing the cost and time of fixing issues later on.
Continuous Testing is used to ensure that the software is meeting the user's requirements, and that it is functioning as intended.
It involves automating the testing process and using tools to test the software continuously throughout the development cycle.
Continuous Testing enables teams to get feedback quickly and make adjustments to the software as needed, improving the quality of the product and increasing customer satisfaction.

What is Chaos Engineering?

Adrian M. — Sun, 05 Mar 2023 04:13:00 GMT

Chaos Engineering is the practice of testing the resiliency of a system and its ability to withstand random and unforeseen disruptions by deliberately subjecting the system to a wide range of stress scenarios. The goal of this discipline is to identify previously undetected points of system failure in order to correct them or to prevent them in the future.

How is Chaos Engineering Different from Testing?

Traditional testing tends to focus on the specific and required capabilities to be supported by the system. It involves identifying a list of specific conditions and scenarios that must be validated to ensure the system works as desired. In a nutshell, traditional testing and quality assurance focuses on testing known conditions.

On the other hand, the chaos engineering approach is much more exploratory in nature; it seeks to elicit unknown and unexpected behaviors from the system. One way to look at it is to consider chaos engineering as an information gathering exercise which would lead to the creation of future scenarios to be considered in traditional testing. In a nutshell, chaos engineering focuses on previously unknown outcomes.

If traditional testing is likened to verifying the skills of first responders to specific emergency scenarios, chaos engineering is likened to simultaneously subjecting the first responders to hurricanes, earthquakes, and wildfires.

What are some of the Chaos Engineering Approaches?

Chaos Engineering at the Infrastructure Level - Evaluate the resiliency of the production platform by simulating failure of various system components, zones, and instances.

Chaos Engineering at the Network Layer - Evaluate the system behavior in the face of the failure of various combinations of network components and connections.

Chaos Testing at the Application Layer - Evaluate the application’s resiliency by simulating failures of various application components across architectural layers.

Chaos Engineering Best Practices

Minimize the blast radius - While conducting experiments in production, it is vital to have safety measures in place to limit the impact (blast radius) of the chaos tests in order to avoid causing unnecessary impact to the users of the system, the customers..

Create small chaos and then big chaos - Start by applying chaos engineering disruptions to a subset of the system in order to determine the effectiveness of the methods employed. Once successful, expand the chaos to the whole system to determine its overall robustness.

Simulate real world events - Define real-world scenarios which could be catastrophic to your system such as hardware failure, software failure, network failure, entire site outages, experts unavailable, etc.

Automate chaos engineering experiments - Once organizations get good at the chaos engineering discipline, they should automate high-value experiments to occur at random (but regular) intervals.

What are some of the tools available to support Chaos Engineering?

Please be aware that this list of tools is rapidly changing as the chaos engineering discipline continues to evolve and mature. Do your own research of the latest and most effective chaos engineering tools.

Chaos Monkey
Litmus
Chaos Toolkit
Gremlin
ToxiProxy
Swabbie
Conformity Monkey
Spinnaker
Chaos Lambda
Pumba
Service Mesh
Istio

What is a Bathtub Curve and How Can it Be Useful?

Adrian M. — Mon, 11 Jul 2022 04:20:00 GMT

Have you heard the phrase...?

"If your project is leaking, check the bathtub."

In engineering and product development fields where the reliability of a product is critical, the bathtub curve is more than a topic of conversation to be had with your interior designer or at your local home improvement store.

The bathtub curve is a graphical model or visual representation of the failure rate of a product over time. In general, one would expect the failure rate to vary over the lifetime of a product, as follows:

early in life of the product - higher-failure rates which are decreasing over time, as issues are fixed and addressed - known as early failures

core life of the product - a more level/constant/predictable failure rate known as random failures

late stage of the product's life - increasing rate of failures, generally different types of failures than in the early stage - known as wear-out failures

If a product's observed or reported failures are plotted on a graph, over time, the result looks like bathtub hence the name: bathtub curve.

Image Source: Wikimedia Commons

The early failures, also known as "infant mortality failures" are expected to exist but are also expected to rapidly decrease as product issues are quickly addressed and remediated.

Wear-out failures are also expected to occur but only towards the end of the projected life of a given product.

How does the Bathtub Curve apply to software projects? How can it be useful?

The bathtub curve can be a useful tool to plot and visualize the expected quality of the software product, system, or sub-module but tracking the failure rates over time and assessing whether the observed failure rates are in line with the expected failure rate.

If the actual failure rates are higher than anticipated, it can signal a flawed development process or a subpar requirements definition practice.

A rapidly decreasing failure rate in the early stage may be the result of a mature and agile development shop where the product is thoroughly tested and immediately rectified as soon as failures are found.

A slowly decreasing failure rate may mean that the product is poorly designed and expensive to fix - begging the question as to whether the product should simply be abandoned.

For the bathtub curve and failure rate analysis to be useful tools, the team must first:

define failures - for software projects, a failure is not just a defect but any unexpected behavior or expected behavior which is not there. Defining failure rates can be a complex endeavor as no two failures are equal. Examples of failures: a traditional bug, product failing to scale to the desired number of users, missing critical capabilities, lacking usability, etc.

track failures - once failures are defined, they need to be diligently tracked in order to have an accurate view of the quality of the product.

One last quick note...

Do you wonder how a software product or platform can experience wear-out failures towards the end of the bathtub curve?

Quick Answer: Have you used Microsoft Windows on a machine for a long time?

Longer Answer: One of the biggest reasons for wear-out failures in software products are the rapidly changing needs and expectations of its users. Consider a smart phone which was created 5 years ago and which still works as well as it did 5 years ago - that is, nothing changed in terms of performance or features. For today's user - who expects a bigger screen, a faster processor, a longer battery life, more features, less weight - the 5 year old phone is worn-out it has many "failures".

As designers of software products (or any products) our goal is to increase the "useful life" of the product, the period in the middle of the bathtub curve where the defect rate is steady and not increasing.

How can we do that?

Here are some ideas:

try to anticipate how the needs of the users might change over time,

design the system to be configurable so that it can be easily changed as the user needs change,

build extensibility into the platform so that new features and capabilities can be added over time,

design the system to be modular so that obsolete capabilities can be easily removed or hidden.

What are some of the questions a business analyst should ask to determine the most appropriate way to Measure Solution Performance (a key task in the BABOK Solution Evaluation knowledge area)?

Chris Adams — Tue, 01 Feb 2022 21:23:00 GMT

The following are the types of questions to ask when determining how to measure solution performance. These will provide the business analyst guidance on how to support Solution Evaluation.

Describe in general terms how a Business Analyst supports Solution Evaluation.

Chris Adams — Mon, 08 Nov 2021 16:23:00 GMT

The interviewee should be familiar with how the Business Analyst supports Solution Evaluation. The IIBA BABOK Solution Evaluation knowledge area includes a number of tasks to evaluate a solution but is not the only source. The interviewee should demonstrate a good understanding of how the BA participates in Solution evaluation.

Measure Solution Performance, Analyze Performance Measures, Assess Solution Limitations, Assess Enterprise Limitations, Recommend Actions to Increase Solution Value

What types of testing should be considered when developing an application?

Chris Adams — Sun, 12 Sep 2021 14:50:00 GMT

The most common types of testing used by a majority of projects are: Unit Testing (Component Testing), Integration Testing, User Acceptance Testing, Functional Testing (Black Box Testing), Usability Testing, Performance Testing (Load Testing, Stress Testing), Regression Testing

Is it a Defect or is it an Enhancement? How to manage the disagreements.

Chris Adams — Tue, 06 Apr 2021 19:25:00 GMT

“It is a defect, or an enhancement” is one of those arguments in software development that never seems to satisfy anyone when someone makes a decision. I’ve gone so far as to say it is a defectment. It’s really a combination of both. However, that doesn’t necessarily answer the question - nor provide guidance to a resolution.

How do you approach user acceptance testing?

Chris Adams — Tue, 04 Aug 2020 20:16:00 GMT

There are three things that are important to consider when preparing for user acceptance testing: 1) identifying the right testers, 2) identifying the right test scenarios, and 3) identifying your testing and communication strategy.

What is the purpose of a Threat and Risk Assessment (TRA)?

Chris Adams — Sun, 12 Jul 2020 17:00:00 GMT

A Threat and Risk Assessment analyzes a software system for vulnerabilities, examines potential threats associated with those vulnerabilities, and evaluates the resulting security risks.

What are the four fundamental methods of requirement verification?

Chris Adams — Sun, 08 Sep 2019 16:00:00 GMT

The four fundamental methods of verification are Inspection, Demonstration, Test, and Analysis. The four methods are somewhat hierarchical in nature, as each verifies requirements of a product or system with increasing rigor.

What type of testing might involve the business analyst?

Chris Adams — Sun, 10 Jun 2018 16:30:00 GMT

Of the various types of testing, the business analyst is usually most involved with Functional Testing, Regression Testing, and Usability testing.

How would you conduct usability testing for a new custom-developed software system?

Chris Adams — Mon, 15 Jan 2018 02:25:00 GMT

Usability testing is conducted in different ways over the software development life cycle:

Usability exploration is performed in the very early stages of software design.

Usability assessment is performed as part of unit and system testing as software is developed and released.

Usability testing is conducted at key development checkpoints, and may be part of an iterative user acceptance testing approach.

What is Gherkin and how can it help the business analyst?

Chris Adams — Mon, 09 Oct 2017 17:52:00 GMT

Gherkin is a structured natural language that is used by business analysts to specify how they want the system to behave for given scenarios. The Gherkin language is simple. It uses about 10 keywords (Given, When, Then, And, But, Scenario, Feature, Background, Scenario Outline, Examples) which allow the language to be read and parsed by an automation tool called Cucumber.

Describe the life cycle of a User Story?

Chris Adams — Mon, 09 Oct 2017 15:20:00 GMT

User Stories are used by agile methodologies to capture the functionality that a system or software should support. For details about what a user story is and how to write one reference What are User Stories.

How can the acronym INVEST assist the analyst during the development of user stories?

Chris Adams — Sun, 12 Feb 2017 11:56:00 GMT

INVEST is an acronym that can help a Product Manager or Developer create quality user stories. INVEST stands for Independent, Negotiable, Valuable, Estimable, Sized-Appropriately, Testable.

I - Independent: The user story should be self-contained if at all possible to avoid dependencies on other user stories. Since one characteristic of agile methodologies is the ability to be flexible and re-prioritize what’s important, independent user stories allow for flexibility during iteration planning. If you do find that your user stories are dependent upon one another, you may be able to combine smaller user stories together that have a dependency between one another. Similarly, you can divide larger dependent user stories into smaller stories such that one of the new smaller stories contains and isolates the overlapping portion of the larger stories.

N - Negotiable: User stories can always be changed or rewritten up until the point of coding. This further supports the flexibility associated with agile methodologies. Since requirements often evolve or rise and fall in priority, user stories should be able to adapt with the changing requirements.

V - Valuable: A user story represents a goal of an end user or purchaser and should deliver functionality that is deemed valuable. This means that specifics of the technical design are not something that you would document as user stories. However, some technical requirements have a component which is valuable to a user. A user might expect pages to load within 2 seconds. The user story would specify the need for 2 second page load times while the specifics of the physical implementation of this would be left out.

E - Estimable: You should always be able to estimate the size of a user story. Sometimes, developers won’t have the experience required to size a particular situation or needed for a user story. When this occurs the user story can be split into two separate user stories. The first is a “spike” which is where developers do some quick research to determine the feasibility of something or get a better idea of how long it might take to implement the particular feature. The spike is always time-boxed, meaning it is limited to a pre-defined amount of time. The “spike” user story might be named “Research (something) to determine…)”, while the second user story is where the functionality will actually be delivered. These two user stories should be scheduled into two separate iterations such than the spike can be completed and the feasibility of the second user story assessed before coding begins. This gives the team time to react if problems arise from the spike.

S - Sized Appropriately: User stories shouldn’t be too big or too small. So how do you decide what size is right. First, any user story that can’t be completed by a developer within a single iteration (or by a developer pair when paired programming is being used) is too big. The user story should be subdivided into two or more smaller stories. Similarly, there is no need to make user stories too granular just for the sake of decomposing features. If features group well together and complement each other then it makes sense to make a single user story. For instance, “As a job seeker I want to be able to add, delete, and edit a job skill on my electronic resume so that I can maintain an accurate listing of my skills.” There is no reason to split “add, delete, and edit” into multiple user stories unless one of them creates a significant amount of work that would make the user story too large for the iteration.

T - Testable: User stories must be testable in order to ensure that development is complete and has been done correctly. So when are user stories not-testable? Often, if the analyst isn’t carful, non-functionality requirements are written in a manner which is un-testable. Consider the example, “pages should always load quickly”. There are two un-testable components of this statement; “always” and “quickly”. A testable statement would be “pages should load within 1.5 seconds 97% of the time”.

--
Chris Adams
LinkedIn Profile

What are User Stories?

Chris Adams — Sun, 13 Dec 2015 16:00:00 GMT

Extreme Programming (XP), one of many Agile methods, introduced the practice of User Stories to describe what a system or piece of software should do. User stories have since been adopted by many of the agile methods used today.

User Stories are short descriptions of functionality that will be valuable to a user or purchaser of the software or application. They describe the users’ goals when using the system. The initial descriptions can be written by the users, customers, product managers, or developers, and are just a few sentences at most (1-3 sentences being typical). This isn’t the entire user story, but it is all that is created at first.

The development of user stories occurs in three parts; the Card, the Conversation, and the Confirmation.

The Card: Named for the standard index cards on which a user story is often captured, Cards include the brief description of the user story, its relative size to other user stories (called story points), and the priority of the functionality. The cards are used for planning the work that will be completed during each iteration of development. If the size of the user story gets too big to complete within a single iteration then it should be broken into smaller stories. The term used to describe a user story which needs to be further broken down into smaller stories is an “Epic”.
The Conversation: While the conversation itself is not an actual deliverable, it is a critical step in the user story development process. Discussions about each user story are had with the users/customers of the system to flesh out details. The details of the conversations are documented in the form of acceptance tests called “The Confirmation”.
The Confirmation: Acceptance tests are details which are captured from the Conversation that can be used to verify that the user story has been successfully implemented. When index cards are used, the acceptance tests are typically written on the back of the card itself. Acceptance tests can and should be captured whenever they are thought of, however, at the beginning of each iteration there is a defined period of time which is set aside to generate acceptance tests.

Using these three parts, the goal of the user story is to plan which functionality will be developed during each iteration, provide enough detail that a developer pretty much understands what needs to be coded, and provide a means to verify that they have achieved the goal. If the developer needs more details, more conversations are had, the details of which are documented as more acceptance tests.

Here are some sample user stories (the Card) for a job board:

I want to post a resume
I want to search for a job
I want to electronically submit my resume for jobs I like

Some user stories follow a more formal structure than others. One formal approach suggested by Mike Cohn follows the structure: “As a (role) I want (something) so that (benefit)”. At first, structuring your user story descriptions like this may seem like overkill sometimes, but it makes sure that you aren’t forgetting WHO you are designing the functionality for and WHY.

As a job seeker I want to post my resume so that recruiters and employers can find it.
As a job seeker I want to search for a job so that I’m in control of my job search.
As a job seeker I want to electronically submit my resume for jobs I like so that I increase the changes of receiving an interview.

Here are some acceptance tests for the user story, “I want to search for a job”

Test with keyword, salary, and location search parameters
Test that the search results are returned in 2 seconds or less

Some comparisons can be made between user stories and use cases, but there are key differences that should be remembered.

Size and Scope	User Stories have limited scope to fit within an iteration.	Use Cases are almost always larger in scope than user stories.
Structure	User Stories typically represent a single scenario or path through a use case. This could be the main scenario, or an alternative or extension path. Remember that the user story includes the acceptance tests which often describe the details covered in alternative and extension flows.	Use Cases represent a series of related user scenarios. While a main scenario (often the most common scenario) is selected, there are many decision points throughout the flow that branch into alternative or exception flows.
Purpose	User Stories are created to facilitate conversation between the client and development team when the time is right, and have the primary purpose of supporting release and iteration planning process. They are never referred back to as a contract between teams.	Use Cases are written to be understood by both the client and the technology team. They represent a written contract of the desired functionality.
Completeness	User Stories are intentionally written at a goal level initially with just enough detail to describe the user story with just a few sentences at most. Only once the iteration planning begins and more detail will be required the team has conversations to capture acceptance tests.	Use Cases are completed in their entirety early in the analysis and design process. Because of this there exists a natural urge by the customers to place screen specific elements in the use cases themselves, even though there is usually a very strong push by the technology team to try and avoid this. Inevitably the technology team rarely succeeds in keeping UI features out of the use cases.
Longevity	Typically User Stories are not intended to live beyond the iteration in which they are developed. Once the functionality has been developed they are discarded.	Use Cases are often saved and become permanent artifacts representing a permanent contract between the customer and development team.

--
Chris Adams
LinkedIn Profile

Describe the V-Model

Chris Adams — Sun, 13 Sep 2015 19:27:00 GMT

The V-Model is essentially an extension of the waterfall model. The system design and development stages of Requirements Gathering through Coding are shown as the downward sloping left side of a “V”, with the Testing stage being broke into its constituent phases of Unit Testing, Integration Testing, System Testing, and User Acceptance Testing creating the upward sloping right side of the “V”. Each system design and development stage on the left corresponds with a testing stage on the right. Each phase of testing is planned in parallel with each of the corresponding system design and development stages to ensure that the appropriate effort and attention is allocated.

When implementing a purchased solution should your requirements and design documents cover features that were not requested but come with the package?

Chris Adams — Sun, 07 Sep 2014 18:22:00 GMT

When implementing a purchased solution at a minimum each additional feature needs to be assessed to determine its potential positive or negative impact.

What is conformance testing?

Chris Adams — Fri, 08 Aug 2014 17:43:00 GMT

Conformance testing verifies that a software product complies with a given standard or protocol. Testing focuses solely on adherence to the rules defined within the standard or protocol. The organization responsible for publishing and monitoring the standard may also provide guidelines or instructions for conformance testing.

The type of conformance tests required will depend on the nature of the standard, as well as the specifics of the associated rules and compliance criteria. For example, conformance testing for a data exchange standard may include verification of data formats, error handling for given scenarios covered under the standard, and may even include tests of data edits and validations used to collect the source data before exchange.

--
Sandy Lambert
Business Architect
LinkedIn Profile

What is fault-injection testing?

Chris Adams — Sun, 13 Jul 2014 16:11:00 GMT

Fault-injection tests involve the deliberate introduction of faults to test system robustness and error-handling capabilities. Faults can be introduced directly into the code (compile-time injection) or through the use of software triggers that cause specific scenarios to occur in a running system (runtime injection). It ensures that the system is able to handle and recover from fault or error conditions, and identifies design weaknesses where a single fault could potentially be propagated into a severe error or systemic failure.

--
Sandy Lambert
Business Architect
LinkedIn Profile

What are the advantages of fixing bugs early?

Chris Adams — Sun, 06 Jan 2013 18:36:00 GMT

Once a bug is identified there are a number of advantages to fixing it as quickly as possible.

Bugs lead to inaccurate status updates and estimates. It’s difficult to estimate the full impact to the project schedule of outstanding bugs. Inevitably, as the project progresses bug fixes and the identification of new bugs seem to take up a larger and larger percentage of the developers’ time. It’s quite common for projects to near the 85-90% completion mark only to hover around that level for some time as bugs continue to be discovered and fixed.

Discussing bugs is an inefficient use of time. Tracking bugs, prioritizing when they should be fixed, and determining how much time it will take to fix them is non-productive. The longer bugs remains unfixed the more time is spent reviewing them during each planning and status meeting.

Unfixed bugs hide other bugs. You can fix one bug only to discover another that wasn’t apparent until the first bug was fixed. Even worse, the new bug that you uncover may be more severe than the original bug and now it’s being fixed much later in the development process.

Bugs make it hard for a team to know if they are reviewing unfinished work or if a bug really exists. Since testing occurs at all stages of development, testers and users often will be reviewing unfinished features within and application. As they stumble across bugs they must try to assess whether it’s an actual bug or whether its just a feature that isn’t fully coded yet.

Unfixed bugs suggest to the team that quality isn’t important. By allowing unfixed bugs to linger and not placing an appropriate emphasis on them, the root causes of bugs (poor specs, poor coding) persist. This creates an environment where mediocrity becomes the status quo.

Unfixed bugs misappropriate the time spent by testers. Good developers should be constantly unit testing their own code. However, when mediocre work becomes the status quo too much is left for the testers to discover. This moves the attention of the testers away from finding the more difficult and complex bugs to finding surface level bugs that should have been fixed prior to QA. Ultimately, this causes testing fatigue and increases the chance the testers will fail to uncover other more complex bugs.

Fixing recent and familiar code is easier for developers. If the developer hasn’t touched the code in several months and then needs to return to it to fix a bug, it takes more time to re-familiarize themselves with the code compared to fixing the bug shortly after the feature was coded.

All of these reflect the advantages of fixing bugs early and quickly.

Describe what is meant by White Box Testing.

Chris Adams — Mon, 12 Nov 2012 01:14:00 GMT

White Box Testing is a form of software testing that focuses not only on what the system should do but more specifically HOW the software will internally accomplish it. The tester carefully chooses inputs that will exercise specific paths through the code in order to produce the expected output.

White box testing can be used during unit, integration, and systems testing, though it is most common during unit testing.

Unit Testing - internal code paths are tested within the unit

Integration Testing - internal code paths are tested which cross between units

System Testing - internal code paths are tested which cross between subsystems.

Because white box testing requires a knowledge of the internal structures of the code, business analysts typically do not participate in this type of testing.

Other names for white box testing include transparent box testing, glass box testing, clear box testing, and structural testing.

What is Black Box Testing and when is it used?

Chris Adams — Sun, 07 Oct 2012 18:15:00 GMT

Black Box Testing is a form of software testing that focuses solely on WHAT the system should do and not HOW the software should internally accomplish it. No specific knowledge is needed of programming or the internal structures of the code. The tester is only aware of what behavior or output should result from a given set of inputs.

Black box testing is one of the most common types of testing in which a business or systems analyst will participate. Test cases are developed from the software specification of requirements. The test cases will state that for a given set of inputs a specific set of outputs should be returned or that a specific outcome should occur.

Black box testing can be used to test for functional and non-functional system requirements, though the majority of black box testing focuses on functional requirements.

Black box testing is used during Unit, Integration, System, and Acceptance testing.

What are the characteristics of a good requirement?

Chris Adams — Sat, 07 Nov 2009 20:11:00 GMT

While different organizations and authors may describe a slightly modified list, the following characteristics are generally accepted as those defining a good requirement.

Cohesive, Complete, Consistent, Modifiable, Correct, Observable, Feasible, Unambiguous, Verifiable

Quality Assurance versus Quality Control

Chris Adams — Thu, 08 Jan 2009 04:32:00 GMT

Quality Assurance is about Process. It describes the proactive method of establishing a process that is capable of producing a product or deliverable that is error or defect free. Expanding on this further, we can see that:

Quality Assurance means time is spent upfront planning and designing a process that can repeatedly produce a high quality product or deliverable.
Quality Assurance provides confidence that if the process is properly followed, there is a high likelihood that the final product or deliverable will meet specifications. In other words, it reduces and prevents defects or errors in the final product or deliverable.
Quality Assurance is about establishing a sound and capable process.

Quality Control is about Products or Deliverables. It describes checking a final product or deliverable to ensure that it is defect or error free and meets specifications.

Quality Control occurs after the process is completed.
Quality Control is about verifying that the product or deliverable meets specifications. It detects errors and defects in the final product before the product reaches the customer.
Quality Control focuses on the end product or deliverable.

To summarize, Quality Assurance is focused on the process, is proactive, and prevents defects; while Quality Control is focused on the product or deliverable, is reactive, and detects defects after the fact.